Applications involving the transmission of video signals are becoming ubiquitous. Industry is on the threshold of developing a wide range of new video applications, such as electronic newspapers, mobile multimedia communications, multimedia databases, multimedia electronic mail, multimedia videotext, remote sensing, sign language captioning, and videophone. Because these applications require the transmission and storage of video signals, effective digital compression techniques are essential to their successful operation.
To guide the advancement of such video delivery applications, the International Standards Organization (ISO) established the Moving Pictures Expert Group (MPEG) to develop compression standards for audio and video data as part of a broader effort to standardize image and video compression. While these standards clearly defined the decompression process, great latitude remained for creative implementations of the video encoder that compressed the video signals. Consequently, industry has produced a variety of video encoders with the objective in mind that a purpose of the video encoder is to code the video signals at a bit rate that does not degrade the perceived quality of the video signals when the signals are decoded and displayed.
Further, the industry has produced several rate control techniques for controlling the bit rate of the video encoder. Examples include the MPEG-2 test model, the MPEG-4 verification model, and the H.263 test model. Most of these techniques focus on the problem of two-way communication, e.g., real-time conversational and interactive services. For two-way communications, coding delay is typically in the order of 100 milliseconds (ms), and, consequently, the size of both the encoder and decoder buffer is small. Because the small buffer size cannot tolerate large bit rate fluctuations, these techniques require video encoders that can tightly control the bit rate. Typically, such techniques achieve the desired bit rate by varying the quantizer step size and by dropping frames when the bit level in the encoder buffer threatens to overflow.
At low bit rates, however, the buffer constraint on the encoder often induces unacceptable spatial distortion and temporal resolution, especially in complex regions of the video. Even when the encoder uses the coarsest quantizer step size, it is still difficult to avoid overflowing the encoder buffer without arbitrarily dropping frames. Moreover, such rate control techniques, in general, do not offer a more comprehensive method for controlling the frame rate other than by resorting to dropping frames.
There remains a need, therefore, for a method and apparatus that can achieve high video quality under low bit rate constraints through a comprehensive balance of trade-offs between temporal resolution and spatial quality while encoding the video signals.